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' Abstract 

, We unify standard frameworks for approachability both in fuh or partial nionitor- 

' ing by defining a new abstract game, called the purely informative game, where the 

outcome at each stage is the maximal information players can obtain, represented 
as some probability measure. Objectives of players can be rewritten as the conver- 
gence (to some given set) of sequences of averages of these probability measures. 
We obtain new results extending the approachability theory developed by Blackwell 
, moreover this new abstract framework enables us to characterize approachable sets 

' with, as usual, a remarkably simple and clear reformulation for convex sets. 

Translated into the original games, those results become the first necessary and 
' sufficient condition under which an arbitrary set is approachable and they cover and 

extend previous known results for convex sets. We also investigate a specific class 
of games where, thanks to some unusual definition of averages and convexity, we 
^ , again obtain a complete characterization of approachable sets along with rates of 

On ' convergence. 
O , 

■ Introduction Repeated games can be studied by considering sequences of payoffs 

and constructing, stage by stage, strategies with the requirement that the outcome at 
the next stage will have good properties given the past ones. Perhaps, the most revealing 
^T) • examples of this claim are Shapley's [TB] operator that describes the value of stochastic 

zero-sum games, the exponential weight algorithm for predictions with expert advices (see 
e.g. Cesa-Bianchi and Lugosi |10) . Chapter 6) or BlackweU's [6] approachability theory. 
^ ' We recall that in a two-person repeated game with vector payoffs in some euclidian 

;h ■ space R'^, a player can approach a given set E C M!^, if he can insure that, after some stage 

and with a great probability, the average payoff will always remain close to E. When 
both players observe their opponent's moves (or at least the payoffs), Blackwell [6] proved 
that if E satisfies some geometrical condition - E is then called a B-set then Player 1 
can approach it. He also deduced that either Player 1 can approach a convex set or 
Player 2 can exclude it, i.e. he can approach the complement of one of its neighborhood. 
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In the partial monitoring case, when players do not observe their opponent's moves 
but receive random signals (their laws may depend on the actions played), working on 
the space of unknown payoffs might not be sufficient - except for specific cases, such as 
the minimization of external regret as did Lugosi, Mannor and Stoltz |16| . 

Attempts were made to circumvent this issue, notably by Aumann and Maschler [1] 
and Kohlberg |14| in the framework of repeated games with incomplete information. 
Lehrer and Solan |15| also considered strategies that are defined, not as a function of 
the unknown past payoffs, but as a function of the past signals, and they proved the 
existence of strategies that satisfy an extension of the consistency property. Perchet |17| 
also used this approach to provide a complete characterization of approachable convex 
sets; it extends Blackwell's one in the full monitoring case. 

Games with partial monitoring 

Formally, we consider a two player repeated game F with partial monitoring where, at 
stage n G W, Player 1 chooses an action z„ in a finite set / and, simultaneously. Player 2 
chooses jn G J. This generates a vector payoff pn = p{in,jn) S IR'^ where p is a mapping 
from / X J to R^, extended to A(/) x A(J) by p{x,y) = E^^y[p{i,j)] := Xiyjp{i,j), 
where A(/) and A( J) stand for the sets of probability measures over / and J. 

The important difference with usual repeated games with full monitoring is that, at 
stage n. Player 1 does not observe Player 2's action jn, nor his payoff pn, but he receives a 
random signal s„ E 5 (where S is the finite set of signals) whose law is s{in,jn) S A(5). 
The mapping s : / x J — > ^{S), known from both players, is also extended to A(/) x A( J) 
by s{x,y) = 'Ex,y[s{i, j)] G A(S'). On the other hand. Player 2 observes in, jn and 

In this framework, a strategy a of Player 1 is a mapping from the set of past finite 
observations := Un6lN(-^ ^ '^)" ^(-^)) similarly, a strategy r of Player 2 is a 
mapping from := UnelN(^ x S x J)" into A(J). As usual, a couple of strategies {a, r) 
generates a probability, also denoted by Po-.r on H°° := (/ x 5 x J)°° endowed with the 
cylinder topology. 

We introduce the so-called maximal informative mapping s from A(J) to A(5)^ by 
^(.y) — {^{hy))i£i- Its range S C A(S)^ is a polytope (i.e. the convex hull of a finite 
number of points) and any of its element is called a flag. Whatever being his move. 
Player 1 cannot distinguish between two actions yo,yi S A(J) that generate the same 
flag p £ S, i.e. such that s(yo) = s(yi) = p, thus s(y) - although not observed - is 
the maximal information available to Player 1, given y E A(J). Note that with full 
monitoring, a flag is simply the law of the action of Player 2. 

Approachability 

Given a closed set E C and 5 > 0, we denote by d{z, E) = infeg_B 11-2 — e|| (with || • || the 
Euclidian norm) the distance to E, by = {cj G R^; d{ijj, E) < 6} the (5-neighborhood 
of E and finally by IIe{z) = {e £ E; d{z, E) = \\z — e\\} the set of closest points to z in 
E (called projections of z). Given any sequence {aml^giN and n > 1, an = ^ Sm=i 
is its average up to the n-th term. 
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Blackwell [6] defined approachabiUty as follows. A closed set E C IR*^ is approachable 
by Player 1 if for every e > 0, there exist a strategy a of Player 1 and N €K, such that 
for every strategy r of Player 2: 

sup E^,^ [d{p^,E)] < £. 

n>N 

In a dual way, a set E is excludable by Player 2, if there exists 5 > such that the 
complement of E^ is approachable by Player 2. 

In words, Player 1 can approach a set E C ^ if he has a strategy such that the 
average payoff converge^ to E, uniformly with respect to the strategies of Player 2. 

In the case of a convex set C, Blackwell [6] and Perchet |17| (see also Kohlberg 
[14j for a specific case) provided a complete characterization with, respectively, full and 
partial monitoring. Those results can be summarized thanks to the following notation 
(that will furthermore ease statements of generalized results). Let X and X denote some 
informative actions spaces and P be a multivalued application from X x X io R'^. In 
our case, X = A(/) and X = S and P{x,^) = \^p{x,y)\ y G s~^(^)} for every x (z X 
and ^ G Af; with full monitoring, X reduces to A(J) and P{x,y) = {p{x,y)}. 

Both conditions of Blackwell [6] and Perchet |17) reduce to the following succinct 
one: 

A convex set C is approachable by Player 1 if and only ifV.^ £ X , 3 x £ X, P(x,^) C C. 

(1) 

Actually, and as we shall see later, this result holds when X and X are any convex 
compact sets of some Euclidian spaces and P : X x X ^ R'^ is a L-Lipschitzian convex 
hull. The latter condition means that for every x G X, P{x,-) is convex and that there 
exists a family {p,^ : X x X ^ B,*"'; k G /C} of L-Lipschitzian functions such that, for 
every (x,^) £ X x X, P(x,^) = co {pk{x,C)] ^ G where co(-) stands for the convex 
hull. 

Purely informative game 

The introduction of two arbitrary compact sets X and X, endowed with the weak-* 
topology, and a multivalued mapping P motivate the following definition of the abstract 
purely informative game T. At stage n G IN, Player 1 chooses x„ G A{X), the set of 
probability measures over X and, simultaneously. Player 2 chooses ^„ G A{X). Those 
choices generate the outcome (the term payoff will only be used in T): 

On = 0(x„, ^„) = x„ (g) ^„ G A{X X X), 

where ® stands for the product distribution. A strategy a of Player 1 is now a mapping 
from UnGiN i^i-^) ^ A.[X))^ to A(X) and similarly, a strategy r of Player 2 is defined 
as a mapping from IJneiN i^i-^) ^ A(Af))" to A(A'). With these notations, a pair of 
strategies (o", r) induces a unique sequence {'X-n-,i,n)n&M (^(^) ^ A.{X))^ . 

^The almost sure convergence can also be required, but to the cost of cumbersome notations. 
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Let 9n, be the average up to stage n of the measures O^i wliicli is defined as follows; 
for every Borel subset F C X x X , OniF) := ^ Em=i ^m{F). Then a closed set E C 
A{X X Af) is approachable by Player 1 if for every e > there exist a strategy a of 
Player 1 and G IN such that for every strategy r of Player 2: 



where W2 is the (quadratic) Wasserstein distance - defined later in a section devoted to 
preliminaries -. For the definition of approachability, any distance metrizing the weak-* 
convergence of measures could be suitable but for the characterization of approachability 
- as we will demonstrate throughout the paper - the quadratic Wasserstein distance is 
very convenient. 

Organization and main results 

The purely informative game F unifies the framework of both games with or without 
signals; we prove indeed in the first section (see Proposition [1]) that a set is approachable 
in a game F if and only if its image set is also approachable in F. And we exhibit in 
the second section, see Proposition [21 a necessary and sufficient condition under which 
the latter holds. So this gives immediately the same result for (non necessarily convex) 
approachable sets with partial monitoring, for the first time in the literature. 

We investigate the case of approachable convex sets, which usually benefits from a 
quite simple characterization (condition ([1])). We show in Section 3 that this is covered 
by our first main result. Theorem [3]- that is actually more general. 

In the last section, we are interested in specific games (that we called convex) and 
thanks to a totally different notion of approachability (along with some unusual definition 
of convexity) we obtain, not only another characterization of approachable sets, but also 
rates of converges. Those main results are stated in Theorem |4] and Theorem [5l 

1 Some preliminaries on Wasserstein Distance and on Nor- 



Here we define in a precise and concise way the distance W2 we have already used in 
the introduction. We also introduce some material that will be used in the sequel. The 
reader can refer for this part to the books |2H 111) . Moreover the projection onto a 
nonconvex set in M'^ is in general multi valued and a proper notion of normal should 
be used (cf for instance |2]). So we need to adapt this definition to the set of measures 
(following ideas of [9]). 

For every /i and u in (R^) - the set of measures with a finite moment of order 
2 in some euclidian space M.^ -, the (squared) Wasserstein distance between fi and ly is 
defined by: 





mals 




sup J{(t),'4)) 



inf En|C/-y|Pl , where 
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- U ^ fi means that the law of the random variable U is fj,; 

- n(//, u) is the set of probability measures 7 G A (R^ x R^) with first marginal fi 
and second marginal 1/ and 

/[7] = / \\x - yfd-f{x,y) ; 

- H is the set of functions (</>, t/j) G L^(]R^, E) x L^(IR^, R) such that (j){x) + V'(y) < 
\\x — yp, /Li (8) i^-as and 

J((/>, V') = / (Ad/x + / ^dz^. 

Furthermore, if the supports of fi and are included in a compact set, then we 
can assume that H is reduced to the set of functions {(j),<j)*) such that, for some 
arbitrarily fixed xq £ K, 

(f>*{x) = inf \\x — y\\'^ — (f>{y), (p = {(f>*)* and <f>{xo) = 0. 

Since every function in H is 2||iir||-lipschitzian (where \\K\\ is the diameter of K), 
Arzela-Ascoli's theorem implies that (H, ||||oo) is relatively compact. 

Actually, the infimum and supremum in ([2]) are achieved; we denote by i/) the 
subset of H that maximizes J((^, (/>*) and its elements are called Kantorovitch potentials 
from ^ to V. Any probability measure 7 G A (R^^) that achieves the minimum is an 
optimal plan from ji to f. 

More details on the definition of W2, based on Kantorovitch duality, can be found 
for example in Dudley chapter 11.8 or Villani |21) . chapter 2. 

Brenier's [8] theorem states that if ji <Co A (i.e., the probability measure /i is ab- 
solutely continuous with respect to the Lebesgue measure A and has a strictly positive 
density), then there exist a unique optimal plan 7 and a unique convex Kantorovitch 
potential from (jl to v. They satisfy: 

dl{x, y) = d/x(x)(5{a._v^(x)} or equivalently 7 = (Id x (Id - V(/))) tj/i, 

where for any ^ : R^ — >• R^, Borel measurable with at most a linear growth, ijj^pL G 
A (R-^) is the push-forward oi lihy - also called the image probability measure of 
by 'i/'. It is defined by 

MK^) = ^ C R^, Borel measurable 

or equivalently by: for every Borel measurable bounded maps / : R^ ^ R: 

/ fdWfi) = [ f{^Pix))d|I{x). 

A classical approximation result (see e.g. Dudley |TT])) is that for any convex compact 
K C with non-empty interior and any e > 0, there exists A^^K) a compact subset 
of Ao{K) = {fj.£ A{K),fj.<^o A} such that, for every ^ G A{K), W^{n,Ae{K)) < e. 
So Brenier's theorem actually implies that 

<I> is singlevalued and uniformly continuous on A^lK) x K. 
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Some geometrical properties of W2 

Blackwell's approachability results rely deeply on the geometry of Euclidian spaces. One 
of our goals is to underline and establish required results for the measure space equipped 
with W2- For instance, in Euclidian (and also Hilbert) spaces the projection onto a closed 
convex set could be characterized equivalently by the minimization of the distance to the 
set or by a characterization of the projection by the well-known condition with scalar 
products. The Lemma [6] could be viewed as a way of writing this "condition with scalar 
products" in the space of measures. Also Blackwell's conditions requires suitable notions 
of projections and normals we will introduce now. 

In the space A(i4r) equipped with W2 we define the corresponding definitions of 
proximal normals (see Bony [7]) to nonempty closed sets A C ^{K) at some fj, £ A. 
As usual, we say that is a projection (with respect to the Wasserstein distance) of a 
measure fi G A{K) if W2{fi,A) := inf^g^ W2{fi,9) = W2(Ai,/f)- 

Actually, proximal normals can be defined in two different ways, depending on which 
equivalent definition of W2 is used. 

- Proximal potential normal: a continuous function (j) : K is a proximal poten- 
tial normal to ^ at /i if is a Kantorovitch potential from to some fi G ^(K) 
where is a projection of fi on A. 

NC^ (/i) = |0:i^— T-M; (f) proximal potential normals to yl at ^u} . 

- Proximal gradient normal (adapted from [1]): a map p G L^j(]R^, R^) is a proximal 
gradient normal to A at /i if p G V'{fJ.,fi) for some fJ, ^ A that projects on ^ and 
some optimal plan 7 G n(/i, /i), where, for every /x, 1/ G A(]R^) and 7 G n(/_f, u), 

V^ifi, /x) = |p G Ll (E^, R^) ; G Clb, [ mx),p{x))dfi{x) = [ (V(x), x - y)d^{x, y) 

with ClB the sets of Borel measurable map ip : — )• with at most a linear 
growth. Riesz representation theorem ensures the non-emptiness of {fi, fi) (see 

NC^ (/i) = |p G L^(]R^,]R^); p proximal gradient normals to ^4 at /i| . 

Observe that Brenier's Theorem also implies that both definitions of proximal nor- 
mals are, in some sense, quite close. Indeed, if j4 is a compact subset of A{K) and £ A 
and n <Co A then 

G NC^(/.) ^ V<^ G NC^(/.). 

2 Equivalences between approachability in both games 

Recall that we represent a game T by two compact convex action spaces X and X 
and a L-Lipschitzian convex hull P. We define the set of outcomes compatible with 
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e G A{X X X) by: 

p{e) = [ P{x,Ode c 

JxxX 

where the integral is in Aumann [3]'s sense: it is the set of all integrals of measurable 
selection of P. For every subset E C A{X x X), the set of compatible outcomes p{E) C 
R'^ is defined by: 

p{E) = OeE^ C 

Reciprocally, for every E C H^', the set of compatible measures p~^{E) C A(X x X) is 
defined by: 

p~^{E) = {ee A(X X X)- p{e) C E] . 

The mapping s does not appear in the description of F but only in the definition of p 
that links F and F: given a set E C R in F, we introduced the image set E = p~^{E) C 
A(X X X) in F. And it is quite intuitive that E is approachable if and only E is (see 
Proposition [1] below, whose proof - mainly technical - is delayed to the Appendix in 
order to keep some fluency). 

Proposition 1 

i) A set E C R'^ is approachable in F if and only if p~^{E) C A {X x X) is approach- 
able in F; 

a) If a set E C A {X x X) is approachable in F then the set p{E) C R'^ is approachable 
in F; 

Hi) For every convex set C C R'^, p~^{C) C A {X x X) is a (possibly empty) convex 
set and for every convex set C C A {X x X), p{C) is a convex set. 

Notice that point ii) cannot be an equivalence. Consider the case p = and E = 
{xq <8> yo} for some xq and yo. Then p{E) = {0} is approachable but E is not, since 
Player 2 just has to play yi ^ yo at each stage. This is a consequence of the usual 
inclusions p [p~^{E)) C E and p"^ {p{E)^ D E. 

3 Approachability of B-sets 

Blackwell [6] noticed that a closed set E that fulfills the following geometrical condition 
- £^ is then call a i?-set - is approachable by Player 1 with full monitoring. Formally, a 
closed subset E of R^ is a i?-set, if 

Vz G R'', 3p G n£;(z), 3x G A(/), {p{x,y) - p,z - p) Vy G A(J). 

An equivalent formulation using NC£;(g), the set of proximal normals to at g, appeared 
in [2]. Indeed E \s a, B-set if and only if 

yp eE,yqe NCs(p), 3x G A(/), Vy G A(J), {p{x,y) -p,q)< 0. 
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Blackwell [6] and Spinat |20) proved that E is approachable in if and only if it contains 
a i?-set. 

Our definition of proximal potential normals gives to W2 a structure close to a 
Hilbert. This allows to extend Blackwell's definition of a B-set as follows. 

Definition 1 A set E C A(X x A') is a B-set if for every 6 not in E there exist 
e G Ii^{e), 4> G ^{6, e) and X (= x(6l)) G A(X) smc/i i/iai ; 

/ d(i-x0^) < 0, V^gA(A:'). 

Or stated in terms of proximal potential normals: 

V^G ^, V(/> G NC^~(^), 3x G A(X), G A(A'), I </> d(^ - x ^) < 0. 
^ Jxxx 

The concept of i?-sets is indeed the natural extension of i?-sets because of the fol- 
lowing proposition. 

Proposition 2 A set E C A (X x X) is approachable if and only if it contains a B-set. 

Proof: We only prove here the sufficient part, i.e. a i?-set is approachable by Player 1 
(by adapting Blackwell [6]'s ideas to our framework). Again, we postpone the proof 
of the necessary part (almost identical to the full monitoring case) to the Appendix to 
prevent cumbersomeness. 

Let e > be fixed. For every probability distribution 9 G A(X x Af), we denote by 
9^ G Afr{X X X) any arbitrary approximation of 6 such that W^(^,0^) < e. 

Consider the strategy cr^ of Player 1 that plays, at stage n G W, x{9f^_i) given by 
the definition of a B-set, where Of^^i is the average of the n — 1 first {9mY ■ Then, if we 
denote by 9^n-i projection of 9^_^i over E and let Wn = W2 (^9f^, L 

Wn = Wi {9'^,e) < Wl = ^^2 + 

sup / (\) d0,_i + / 
<i>&.JXy.X JxxX 



n 



I (t)n dC-1 + / Cd 
JXxX JXxX 



n 
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9^ 
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n 





n \JxxX JxxX 

where is the optimal Kantorovitch potential from O^^i to + ■ Let us denote 

by (po the optimal Kantorovitch potential from 9f^_i to 9^_i and by uj^{-) the modulus 
of continuity of <^ restricted to the compact set E x Ai;{X x X). 
The definition of W2 implies that 



9i\ _ . M\X xXf 



'2 \ ^n-l^ "n-l "I I ^ "''''2 y'^n-li^n 

n n n n 
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therefore \\(j)Q — (pnWoo ^ "^^(^ll^ll/v^) 

Wn < "^^^Wn-l + -( f (Pode^.i + [ (t^ode^n) + j ^H^ ^ A' || \ ^ 

n \JxxX Jxxx J 'n \ J 

Recall that 6^ is such that that Jxxx 4'odOn + JxxX '^o^^n. ^ ^ii^n, On,) < therefore 



n-l Iff , /■ , A 2 f2\\XxX\\\ e 

Wn < Wn-1 + - / 4>odOn_i - / (jyodOn + ^ + -• 

n n Klxs^y . Y^y n \ Jn / n 



Since is a i?-set and because of the choice of x„ G A(X), for every G ^{^X)^ 

JxxX (C-l - Xn <8) ^n) < 0> t^US 

n-1 2 ^ /2||X X A:" ||\ e 

Ifn < Wn-l + 1= + - 

n n \ / n 

and this yields, by induction, that 

m=l \ V / 

Since a;''(2||X x X \\/^/k) converges to when k goes to infinity, then Wn is asymptot- 
ically smaller than e. The fact that W2 (^On,E^ < W2 (^6n,E^ + e implies that E is 
approachable by Player 1. □ 

4 Characterization of convex approachable sets 

There also exists in F a complete characterization of approachable convex sets : 
Theorem 3 A convex set C is approachable if and only if: 

y$ G A{X), 3x G A(X), 0(x, = X » ^ G C. 

Proof: Once again, we will follow the idea of Blackwell. Assume that there exists ^ such 
that, for every x G A{X), 0(x, ^) ^ C. The application x 1— )• W2 ^0(x, ^),C^ is continu- 
ous on the compact set A{X), therefore there exists 6 > such that W2 ^^(x, > 6. 
Consider the strategy of Player 2 that consists of playing ^ at every stages, then 
= x„ ® ^, ^„ = Xn (g) ^ = 6'(x„,^) and (On,C^ > 6 > 0. Therefore C is not 
approachable by Player 1. 

Reciprocally, assume that for every ^ G A{X) there exists x G A{X) such that 
9{x, ^) G C. We claim that this implies that C is a S-set. 
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Let be a probability measure that does not belong to C and assume (for the 
moment) that 9 X. Denote hy 9g C any of its projection then, by definition of the 
projection and convexity of C: 



W2{0, 9) < Wi ((1 - A)^ + Ax ® ^, 0) = sup / (pd ((1 - X)9 + Xx(^$)+ [ (/.M^ 

= sup [ <pd9+ [ 4>*d9-X [ (t)d{9-x(^^) 
= [ (Pxd9+ [ (t>ld9-x[ (AAd(i-x®0 

JXxX JxxX JxxX 

<[ ^od9+ [ 4)ld9-xf (/.Ad(^-x0O 

JxxX JxxX JxxX 

= Wi(0,9)-x[ ^xd{9-^^^) 

JxxX 

where (px (resp. (po) is the unique potential from (1 — A)^+Ax(8)^ (resp. 9) to 6. Therefore, 
for every A > 0, A /jj^xA" 4'xd{9 — x 05 ^) < 0. Dividing by A > yields: 



[ (/>Ad(0 - X (g) ^) < 0, VA>0. 

JxxX 



Since (1 — A)^+ Ax®^ converges to 9, any accumulation point of ((/>a)a>o has to belong 
(for every x and ^) to ^{9, 9) = {^o}- Stated differently, given G one has: 



lin (7^,, (x,f) := max min / (hr)d(9 — :x. (E> < 0- 
\{X) i&A{X)yceA{X)JxxX 

The function g^^ is linear in both of its variable, so Sion's Theorem implies that 



max mm 

ieA(X) x6A{X)' 



max min g^n(x,£) = min max g^n(x,£) 

^GA(X)yceA{X)^^°^ yc&A{X)i€A{X)^^°^ 

hence C is a i?-set. 

Assume now that 9 Ao(^ x X) and let 0„ G Ai {X x Af) be a sequence of measures 

n 

that converges to 9, (^„)„gjN a sequence of their projections, and E $(^„,^„). Up 
to subsequences, we can assume that 9^ and 4>n converge respectively to 9 and i^o- 
Necessarily, ^ is a projection of 9 onto C and (pQ belongs to ^{9,9). Therefore: 

/ (pod{9 - X ® ^) = lim / (Pnd{9 - x ^) < 
JxxX "■^°°JxxX 

and C is a i3-set. □ 

Let us go back and quickly show that the characterization ([TJ of approachable convex 
sets with full monitoring is a consequence of Theorem |3l 
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Proof of characterization ([T]) By Proposition [T] a convex subset C of R'^ is 
approachable if and only if the convex set p~^{C) C A (X x X) is approachable in F. 
Therefore, using Theorem [3l if and only if for every ^ G IS.{X), there exists x G 
such that /3(x (8) ^) C C Let us denote by Ex G X and G A" the expectations of any 
x G A(X) and ^ G A(A'). Then, in the case of games with partial monitoring, one has 
that p{5-E^ ®5^)C. p{x (g) 6^) and p{5x <8) ^) C p{5x <8) (5e^). 

Assume that C is approachable; since the condition holds in particular for ^ = S^, 
there exists x such that p{x 6^) C C, therefore some x £ X such that p{6x ® 6^) C C 
(one just has to take x = Ex). 

Reciprocally, if C is not approachable; there exists ^ such that (in particular) for any 
^x, ® ^x) ^ C, therefore there exists some ^ G Af such that p{6^ ® 5x) <^ C (one just 
has to take ^ = E^). 

We obtain the stated result as a consequence: 

C is approachable if and only if G , 3 x G X, p{5x ® S^) = P{x, ^) C C 

□ 

This also explains why there exist convex sets that are neither approachable, nor 
excludable with partial monitoring (see Perchet |17| ) which cannot occur with full mon- 
itoring (see Blackwell [6]): it simply due to the fact that /3"^(C) can be empty. 

5 Convex games 

We restrict ourselves in this section to the particular class of games called convex games 
which have the following property: for every q G A{X x X): 

p{q) = [ P{x, Odq{x, 6 C P (E, [x] , E, [C] ) . 

For instance, this reduces in games with full monitoring to p{q) = /? (Eq[x], Eg[^]). 

Example 1 The following game where the payoffs of Player 1 are given by the matrix 
on the left and signals by the matrix on the right, is convex. 





L 


c 


R 




L 


C 


R 


T 


(Orl) 


(1,-2) 


(2,-4) 


T 


a 


a 


b 


B 


(1,0) 


(2,-1) 


(3,-3) 


B 


a 


a 


b 



In this game I = {T,B}, J = {L,C,R} and S = {a,b}. If Player 1 receives the signal 
a, he does not know whether Player 2 used the action L or C. 

We introduce the notions of displacement interpolation and convexity (see e.g. Vil- 
lani |21| for more details) that will play the role of classic linear interpolation and con- 
vexity. 
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Given ^, u £ A^(R^) and t G [0, 1], a displacement interpolation between and v 
at time t is defined by Jit = (Tt^'j, where 7 E n(;U, z/) is an optimal plan and at is the 
mapping defined by at{x, y) = (1 — t)x + ty. A set C is displacement convex if for every 
HjI' £ C, every t G [0, 1] and every optimal plan 7 E n(;U, z/), critl7 S C 

Let r be a new game defined as follows. At stage n € M, Player 1 (resp. Player 2) 
chooses Xn S X (resp. y„ € X) and the payoff is On = Sx„ (B) Sy^ = 6{^xn,y„) ^ A(X x X). 
We do not consider average payoffs in the usual sense (as in T) but we define a sequence 
of recursive interpolation by: 

On+i = 0"^tJ7n+i, where 7^+1 G n(6'n,6'n+i) is an optimal plan. 

By induction, this implies that On = &xn ® ^y^ ■ Indeed, Oi = <8' Sy^ and 02 = ^y2 
therefore: 

72 = (i^zi (8) (^^/J (g) {5x^ 18) Sy^) and 02 = C7i(j72 = (5 £1+^2 <8) 5yi+Ji2 • 



Definition 2 yl closed set E C A(X x A') is displacement approachable by Player 1 
if for every e > there exist a strategy a of Player 1 and G IN such that for every 
strategy t of Player 2: 

yn>N,W2(0n,E') <e. 

Consider any set E C R'^ and assume that p~^{E) is displacement approachable 
by Player 1. Since 0^ = 5x„ ® the convexity of the game implies that p{On) C 
p{On) and thus p^^^E) is also approachable in the sense of T. The use of displacement 
approachability provides however explicit and optimal bounds (see Theorem H] below) . 
This is the reason we investigate this special case. 

In this framework, we use proximal gradient normals to define a i?-set: 

Definition 3 A closed subset E C A {X x X) is a B-set if for every not in E there 
exist a projection G Il^{p), p G NP^{0) and x = x{0) G A (X) such that for every 
y £ X, there exists an optimal plan 7(x, y) G n(^, 6x Sy) and p{x, y) G 7^(7(2;, y)) such 
that: 

{p,p{x,y))L^(^g_) < 0. 

This notion of i?-set extends Blackwell's one to F because of the following Theorems U] 
and [5j 

Theorem 4 A set E is approachable in T if and only if it contains a B-set. Given a 
B-set, the strategy described by Xn+i = x{On) ensures that W2 (^On,E^ < Kj^fn, for 
some K > 

Proof: Assume that Player 1 plays, at stage n, Xn = x{On-i) and denote by On = 
5xn ® ^yn outcome at stage n. For every n G IN, the displacement average outcome 

is On = fe„ ® = 5(x^^yJ. 
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If we denote by 6^ G E the projection of 0„ on E, then the optimal plan from 9^ to 
9n is 0_^® On- So the proximal normal p„ G NP^iO^n) is defined by p„(2;) = z — (xn^Vn)- 
Similarly, 9^ (X" (5x„+i,j/„+i) is an optimal plan from 9^ to (^x„+i ® "^s/n+n thus if we define 
Pn+i{z) = z — {xn+i,yn+i), the assumption that is a S-set (along with the choice of 
Xn+i = x{9n)) ensures that (p„,p„,+i)e^ < 0. 

As usual, we note that < (0„4.i,^„) which satisfies: 



(9n+i,9n) = \\x - zf d9n+i 9^ = - z||^d^„(z) 

^ ^ J(XxX)^ Jxxx 



(XxX)^ 

n 



(Xn, Vn) H — r {Xn+l, Vn+l) " Z 



XxX 

n 

n + l 



n+l n+l 

2 



2 



1 ^2 



n+l 



+ { — — T ) / IKa^n+li^n+l) - d^n(^) 



71 / 

+ ^7 r~n2 / {{Xn^Vn) - Z,{Xn+l,yn+l- Z))d9^{z). 

yn + l) JxxX 



Therefore, 



We conclude by induction over n E IN. 

We sketch the proof of the necessary part. Conclusions of Lemma [8] (delayed to 
Appendix) hold in T and the proof of the first two points are identical. Hence it remains 
to prove the third point, i.e. that a set which is not a S-set has a secondary point (see 
Definition H] also in Appendix). Let 9 be not in i?, ^ one of its projection on E, and 
p G NC^(0 the associated proximal normals such that: 

Vx £X,3y£ X, {p,p{x, y))^ = / {p{z),z - (x, y))d9_ > 0. 

JXxX 

Sion's theorem implies the existence of 5 > and y £ X such that for every x G X, 
{p,p{x, y))0 > 6. If we denote by 9x = (Id, ax) tt^„ <Si 9n+i then using the same argument 
as in the proof of Lemma [6l we show that 

W2(9,9x) < W2{9,9) + ^^^^ '-^^^ < W2(9, 



2W2{9,9) - ' 2W2i9,9y 
for A small enough. Hence, ^ is a secondary point. □ 

The following Theorem is the characterization of displacement convex approachable 
sets. 
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Theorem 5 A displacement convex set C is approachable by Player 1 inT if and only 

yyeX,3xeX, 9{x,y) = 6x0Sy e C. 

The proof is based on the following lemma that extends to Wasserstein space the usual 
characterization of the projection on a convex set in an Euclidian space . 

Lemma 6 Let X be a compact subset of and A be a displacement convex subset of 
A{X). Fix 9£ A. Then for all p £ NC^(^) and all 9i £ A we have 



yper{e,ei), / {p{x),p{x))de{x) ■.= {p,p)e <o. (3) 

Proof: Let us consider 6,6q £ A and p G NP^{6). We denote hy 6 ^ A the measure 
outside A and 7 G n(^, 6) the optimal plan given by the definition of proximal gradient 
normals. Define 7' = Ttt7 where T : (x, y) 1— )■ {y, x) so that 7' is obviously an optimal 
plan from 9 to 9. 

Let 7 G n(^, ^o) be an optimal plan from 9 to 9q and for any A G [0, 1] we define 
:= CAttT ^-iid 7a = (Id, (J\)^J which belongs respectively to the displacement convex 

set A and to U{9,9x). 

By the disintegration of measure theorem for any y G there exists a probability 

measure ^x,y on such that ^x = /]Riv('^j/ l\,y)^{dy) which means that for any 

continuous bounded function u{y, z) : 1— t- R 



u{y,z)-ix{dyAz) = 

R2iV J-^N 



U{y,z)-fx,y{dz) 



0{dy) 



We define 7 G n{9,9x) by: 



eCb, 4>dj = 4>{x,z)-fx,y{dz)j'{dx,dy). 

yR2^V Jr3N 



Since 9x £ A and 7 G 11(0, 9x), we obtain: 
W^{9,9) < W^{9,9x) < [ \\x- zfd^ 



X - zf^x,y{dzW{dx,dy) 

\\x - yf^x,y{dz)'y\dx, dy) + 2 {x - y,y - 2;)7A,j;(dz)7'(dx, dy) 

+ / \\y- z\\'^7x,y{dz)'y'{dx, dy) = a + b + c 
Jr3Jv 

where a, b and c denote respectively the three integral terms in the above equality. It 
remains to estimate the three terms a, b and c. 



/ \\x-yfj'{dx,dy) = W^{9,i 

Jr2JV 
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b = 2 



(x-y, {y- z)-fx,y{dz) ) -y'idx, dy) 
2 (y — x, {x — z)^x x{dz) ) ^{dx,dy) (by definition de 7') 
—2 / (p{x), / {x — z)jx x{dz) ) 0,{dx) (from the definition of ^) 



= -2 {p{x),x- z)jxA^z)e{dx) 

Jr2N 

= —2 I {p{x),x — z)^x{dx,dz) (by the desintegration formula) 

= —2 {p{x),x — [{I — X)x + Xz])j{dx,dz) (by definition of 7a) 

= — 2A / {p{x),x — z)'j{dx,dz) = —2X / {p{x) , p{x)) 9{dx) . 

And this holds for any p G V{6,6q). 

The disintegration of measure formula together with the definition of 7 yield 

c=/ ||y- [(1 - A)y + Az] |Pd7(y,z) = / \\y - z\fd^{y, z) 

Jr2N J^2N 

hence c = X'^Wi {6,60). 

Summarizing our estimates, we have obtained 

w^{e,e) < wi{e,e) -x [ 2 < p{x),pix) > ^(dx) + x'^w^{9,eo). 

Jr^n 

Thus for any A G (0,1), 

O<X^W{0,9i)-2X [ <p{x),p{x) > e{dx). 

Dividing firstly by A > and letting secondly A tend to 0"*", this gives the wished 
conclusion. □ 

Proof of Theorem [5l Let 9 be any measure not in E and denote by ^ G IIq{6) any 
of its projection and p G NPq{6), associated to some 7 G n(^, 9), any proximal normal. 
For every x € X and y G the only optimal plan from OtoSx'Sidy is 7 = {Sx (8) 6y)- 

The function h : X x X defined by 



Hx,y)= < p{u),u - V > dj{x,y) 

J{XxXf 



is affine in both of its variable since: 



Hx,y)= {p{u),u) de{u) - ( p{u)de{u),{x,y)) = z - {z,{x,y)), 

Jxxx \JxxX I 
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where z = / {p{u),u)d9{u) and z = / p{u)d9{u). 
JxxX JxxX 

Since for every y £ X, there exists x £ X such that 5x®6y G C, Proposition [6] imphes 
that for every y £ X there exists x £ X such that h{x,y) < 0. X and X are compact 
sets, therefore Sion's theorem imphes that there exists x £ X such that h{x, y) for every 
y £ X. Hence C is a i?-set and is approachable by Player 1. 

Reciprocally, assume that there exists y £ X such that 5x ® 5y ^ C for every x £ X. 

Since X is compact, there exists 7/ > such that vnix^x W2 (^Sx fX" 6y, > ij. The 

strategy of Player 2 that consists of playing at each stage 6y ensures that On = Sx„ ® Sy 
is always at, at least, 6 > from C. Therefore it is not approachable by Player 1. □ 



Concluding remarks 

Recall that action spaces in T (resp. F) are A{X) and A{X) (resp. X and X). Assume 
now that in T players are restricted to X and X; then a B-set should satisfy: 

G ^, V(/. G Nct{e), 3x £ x,yy £ X, [ (f>d{e-6x(^ Sy) < o. 

^ JxxX 

The proof of the sufficient part of Theorem [2] does not change when we add this assump- 
tion, thus a i?-set is still approachable. However, both the proof of the necessary part of 
Theorem [2] and the proof of Theorem [3] are no longer valid (due to the lack of linearity) . 

Similarly, assume that in T players can choose action in A(X) and A{X) and, at 
stage n G IN, the outcome is On = x„ (g) Strictly speaking, given such outcomes that 
might not be absolutely continuous with respect to A, the sequence of interpolation On 
may not be unique. However, we can assume that the game begins at stage 2 and that 
Oi = X/X{X X X); then, see e.g. Villani |21| . Proposition 5.9, O2 <^ X and is unique. 
By induction, the sequence of On is unique. Once again, using the same proof, we can 
show that a i?-set is displacement approachable, but we cannot extends the necessary 
part nor the characterization of displacement approachable convex sets. 



A Proof of Proposition [T] 

Let us first state and prove the following (implicitly stated) lemma: 

Lemma 7 // the two functions p and s are linear both in x £ A(/) and p £ S, then 
the multivalued mapping {x,fi) 1— ?• P{x,ij) = {p(2;,y); y £ s~^(;u)} is a L-Lipschitzian 
convex hull. 

Proof: Since the graph of is a polytope of R'^^ x JR,"^, there exists a finite family of (so 
called) extreme points functions {?/«;(■)> ^ £ IC} from S into A (J) that are all piecewise 
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linear and continuous (thus Lipschitzian) such that s ^(/i) = co{?/g(/i); kGIC}, for 
every G S. Since p{x, •) is hnear on A(J): 

P{x,n) = {p{x,y); y G s~^(/x)} = {p{x,y); y G co{y«;(/i),K G /C}} = co i^p{x,y^{p)); k G 

Therefore P is indeed a L-Lipschitzian convex hull. □ 

We now turn to the actual proof of Proposition [1) 
Proof of Proposition [T] 

The third point is obvious, so we only need to prove that if C A {X x X) is 
approachable in F then p{E) C R*"' is approachable in T (see part 1) and that \i E d R'^ 
is approachable in T then p~^{E) C A {X x Af) is also approachable (see part 2). The 
remaining easily follows from the fact that p(^p~^[E)^ C E. 

part 1: The proof consists in two steps. First, we link the Wasserstein distance 
between two probability measures 9,9 G A{X x X) and the distance between the two 
sets p{9) C R*^ and p{9) C R^. We will prove this step with the use of the 1-Wasserstein 
distance defined below. In the second step, we transform a strategy in F into a strategy 
in TP. 

Step 1: The 1- Wasserstein distance between p and u in A(X) is defined by: 

Wi(9,e)= inf / ||x-?/||d7(a;,y) = sup / (I)d9-d9= inf E[||C/-y||], 
jeu{e,e) Jx 0eLipi(x,R) Jx [/~e,y-e 

where Lip]^(X, R) is the set of 1-Lipschitzian functions from X to R. Jensen's inequality 
and the probabilistic interpretation imply that Wi{9,6) < W2{9,6). 

Let P is a L-Lipschitzian convex hull, then since P{x, ^) = co \^Pf^{x, k G /C} where 
every is L-Lipschitzian, for every 9 G A{X x X), by convexity of the integral (see e.g. 
Klein and Thompson |13| . Theorem 18.1.19): 

/ Pix,0d9= [ coK(x,0; A^G/C}d^ = co| /" p^ix,0d9; k G ic] . 

JxxX JxxX IJxxX J 



The mapping is L-Lipschitzian, so d \^J^ ^-^ Pfi{x, S^)d9 , p{9)j < VkLe and since the 
set p{9_) is convex, d{p{9),p{9)) < VkLe. Therefore, 

W2 (9, e) <e ^ sup d{z, p{E)) < ^/kLe . 



Step 2: This step transforms a strategy in F into a strategy in F and is quite standard 
in games with partial monitoring (see e.g. Lugosi, Manner and Stoltz |16|): its proof, 
which relies deeply Hoeffding-Azuma's |12[ [5] inequality, is therefore only sketched. 

Let a he a, strategy of Player 1 that approaches (up to e > 0) a set E C A {X x X). 
This strategy cannot be directly played in F in order to approach p{E) for two reasons: 
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1) in r, Player 1 chooses an action i £ I and not some x G A(A(/)); 

2) at stage n in T, the flag fj,n = s{in,jn) is not observed, but only a signal s„ whose 
law is s{in,3n)- 

The usual trick is to divide IN into blocks of length G IN - where is big enough. 
The n-th block in V will correspond to the n-th stage in F and a is defined inductively. 
Assume that a dictates to play x„ G A(X) at the n-th stage of F. In T and independently 
at every stage t of the n-th block, with probability r] the action it is chosen uniformly 
over I, and with probability 1 — r/ accordingly to Ex„ G X. 

The stages where it was chosen uniformly allow to build an unbiased estimator that 
will be arbitrarily close (if A^ is big enough) to 77„, the average flag during the n-th block. 
Since the choice of actions are independent, it is easy to show that the average payoff on 
the n-th block is arbitrarily close to the 77-neighborhood of i-*(Ex„,7x„) = p{Ey_^ , . 
So it is enough to act as if Player 2's action in F was S-p^. 

In order to obtain the almost surely convergence, we can use a classical doubling trick 
argument (see e.g. Sorin [19]), which consists in a concatenation of these strategies with 
increasing A^ and decreasing rj. 

Part 2: Assume that E C M,^ is approachable in F by Player 1. Consider the game 
where Player 1 observes in addition /i„ = s(j„) and his payoff is Wix^lpiin, jn)] where Xn 
is the law of i„. This new game is easier for Player 1 because he has more information 
and actions, hence he can still approach E C R^'. Since P{x, •) is convex, allowing 
Player 2 to play any action in A{X) does not make the game harder for Player 1. Thus 
we can assume that at stage n G IN, Player 1 observes ^„ G A(A:') = A(5), that he plays 
deterministically x„ G A(X) = A(A(/)) and that his payoff belongs to /9(x„ (X" We 
call this new game by F'^. 

The fact that E is approachable in F*^ implies that for every e there exists a strategy 
Us in F'^ and A'g G M such that for every n > and strategy r of Player 2: 

^(i:i=iPi^^^U^^^ :=sup|d(z,£;); ,e^:^=i^(^"-^^>^)|<,. (4) 

If we denote as before 0„ = x„, ^„ G A{X x X) then equation ^ becomes: 
Ve > 0, 3Ne G M, Vn > A^^, 9n G P~\E'). 

Let us define similarly p-^{E)^ = ^6 £ A{X xX); W2 (0,eJ < jj. Since the se- 
quence of compact sets p~^ (E^) converges (as e converges to zero) to p~^{E), for every 
(5 > 0, there exists e such that for every < e < e, p~'^{E^) C p~'^{EY. Therefore, 
for every 6 > 0, there exists A^ G IN such that for every n > N and every strategy r of 
Player 2, 9n belongs to p~^{E)^ . Thus p~^{E) is approachable by Player 1. □ 

B Proof of the necessary part of Theorem [2] 

The necessary part of Theorem [2] is an immediate consequence of Lemma [HI greatly 
inspired from Spinat [20] : it requires the following definition. 
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Definition 4 A point 6 € A{X x X) is 6-secondary for E if there exists a corresponding 
couple: a point ^ E ^i^) o,nd a continuous function A : A(X) — )■ (0, 1] such that 
™i'^xeA(X) ^2 ( A(x)^ + (1 — A(x))x ® ^, j > 5. A point 6 is secondary to E if there 



exists 5 > such that x is 6-secondary to E. 

We denote by VV{E) C E the subset of primary point to E (i.e. points of E that are 
not secondary). 

Lemma 8 (Spinat [20] ) 

i) Any approachable compact set contains a minimal approachable set; 

ii) A minimal approachable set is a fixed point ofW; 
Hi) A fixed point ofW is a B-set. 

Proof: i) Let B = <Z E i?isan approachable compact set| be a nonempty family 

ordered by inclusion. Every fully ordered subset of B has a minorant S (the intersection 
of every elements of the subset) that belongs to B since it is an approachable compact 
subset of E. Thus Zorn's lemma yields that B contains at least one minimal element. 

ii) We claim that if E is approachable then so is VV{E), hence a minimal approach- 
able set is necessarily a fixed point of VV. Indeed, if 9 is 5-secondary there exists an 
open neighborhood V of 6 such that every point of V is (5/2-secondary, because of the 
continuity of W2. Hence VV{E) is a compact subset of E. 

Let ^0 be a (5-secondary point of an approachable set E and ^, A the associated couple 
given in Definition [H Let e < (5/4 and consider a a strategy of Player 1 that ensures that 
9n is, after some stage E M, closer than e to E. We will show that On must be close 
to ^0 only a finite number of times; so Player 1 can approach E\{0~\. Indeed, assume 
that there exists a stage m E IN such that W2{0m, ^o) ^ ^/^ ^-^d consider the strategy of 
Player 2 that consists in playing repeatedly ^ from this stage on. It is clear that (if m is 
big enough) after some stage 6 will be (5/2-closed to A (x„,m) + (1 ~ i^n,m))^n.m <^ C 
where x„,m is the average action played by Player 1 between stage m and m+n. Therefore 

W2 (^6n,E^ > 6/2 > e and since W2{On,Oo) can be bigger than 6/4 only a finite number 

of times, the strategy of Player 1 approaches E\{9}. This is true for any secondary 
point, so Player 1 can approach VV{E). 

Hi) Assume that E is not a B-set: there exists 9 ^ E such that for any projection 
9 E n^(^), any (p E ^{0,9) and any x E A{X), there exists ^ E A{X) such that 
IxxX </'d(^ — x®^) > 0. This last expression is linear both in x and ^, so Von Neumann's 
minmax theorem imply that there exists ^(= ^{9, (j))) and 6 such that j/^xx 't^^i^ — x 
^) > 5 > 0, for every X E A(X). 

We can assume that 9 E A.q{X x X). Otherwise, let {9n E Aj_(X x X))neM be a 

n 

sequence of measures that converges to 9 , (^„)nelN a sequence of projection of 0„ onto 
E and (/)q E ^(^n;^n)- Up to two extractions, we can assume that 0^ converges to a 
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projection of 9 and 0g converges to G <J>(^,^g). Therefore, for n big enough and for 
every x E A(X), 

0<^< / <^Sd(^„-x^5^(^o,<Ao)) 

since the right member converges to Jxxx '^0^ (—0 ~ ^(^O' ^o)) ^ 

For every A G [0, 1] and x E A(X), we denote by (/);^^x the unique (we assumed that 
6 <^o) Kantorovitch potential such that: 

Wi ((1 - A)^ + X^(^tO)= I 0A,xd ((1 - A)^ + Ax ^) + / (I^Ude 



IXxX 



{{1- x)e + xx<g)$,e) = [ 0A,xd((i- A)^ + Ax®o+ / 

JxxX Jx 
= [ </'A.xd^ + / ^l^dO-X [ cl)x,^d{e - X ^) . 

JxxX JXxX JXxX 

Since (A,x) i— t- (px^y: is continuous, (t)\^:x_ converges to (pQ^ for every x G A(X) which is 
compact. Hence there exists AG (0, 1] such that: 



/ (</'A,x-</'o)d(^-x®0 

JXxX 



<5/4, VA < A,Vx G A(X). 



Therefore, one has {O, (1 - A)^ + Ax ^) ) ^ < (W2{e, 9)^ ^ - A| so 

W2 (9, (1 - A)^ + Ax ^5 1) < VF2(^, ^) - z^TTTs^, ■= ^2(0, 9) - r, 

oW2[u,0) 

which imphes that W2 ^(1 — A)^ + Ax (8) > rj and 9 is r/-secondary to E. 

Consequently, a fixed point of W, i.e. a set without any secondary point, is necessary 
a 5-set. □ 
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